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The minimal comprehension principle asserts that the 
act of comprehension must entail an interaction beti^een an incoming 
linguistic message and the reader's world knowledge. An analysis of 
current tests of reading comprehension indicates that test passages 
are likely to draw broadly from knowledge of the world, so that some 
of the variability in test performance must be attributable to 
differences in prior knowledge rather than to differences in reading 
skills. Most tests of reading comprehension currently in use do not 
distinguish between these two sources of variability. This does not 
affect their usefulness as predictive instruments but greatly 
restricts the use of the tests for diagnosis of reading, difficulties 
or assessment of educational gains. Optimal tests for these purposes 
would reduce variability due to prior knowledge or reasoning ability. 
Various suggestions for achieving this control are in need of 
empirical verification. (AA) 
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_„.On''the Theory and Measurement 

^^^-^—^'"^ of Reading Comprehension 

The purpose of this paper is to review recent developments in, the 
theory of comprehension and to derive implications from this work for the 
measurement of reading comprehension. Many recent commentaries on the topic 
of comprehension (e.g., Otto, 1971; Simons, 1971; Stauffer, 1971; R. L. 
Thorndike, 1973-197^; Tuinman, 1971) have lamented the fact that our theo- 
retical knowledge of the comprehension process has not progressed very far 
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beyond the- observations of early reading researchers such as Huey (I908), 
Jamei (1890), Richards (1929) or E. L. Thorndike (1917). While the obser- 
vations of these early theorists contain many insights into the processes 
involved in comprehension, our view is that real progress toward the develop- 
ment of a psychologically valid theory of comprehension has emerged only re- 
cently from attempts to develop process models of human cognitive functioning 

In the paper which follows, we will propose a minimal principle of com- 
prehension which we have derived fxom recent theoretical and empirical work 
on the comprehension process. This principle, we believe, must be part of 
any serious theory of comprehension. In subsequent sections we will review 
several types of models of the comprehension process and show how our mini- 
mal principle is integratoid within them, derive the implications of the mini- 
mal principle for the measurement of comprehension, and .propose alternatives 
to the existing methods of measuring reading comprehension. 
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A Minimal Principle of Reading Comprehension 



The purpose of this section is to present arguments in support of a 
minimal principle of comprehension. Our use of the term "minimal" is self- 
descriptive in that we believe this principle must be part of any serious 
theory of comprehension. The principle is that the act of comprehens i'on en- 
tails an interaction between an incoming linguistic message and the compre- 
hender's world knowledge. This principle can take a weak and a strong form. 
The weak form of the principle suggests that the reader's prior knowledge 

plays a part in the perceptual aspects of the reading process. This would 

I 

include activities such as identifying features in letters, attaching speech 
sounds to particular letter and spelling jjatterns, an\ i dent i fy ing words and 



word meanings. In the weak form of the minimal comprehension principle the 
act of comprehension entails a process o'fi stringing together a sequence of 
derived word meanings until a "unit" (e.g., sentence, phrase, proposition) 
has been achieved and then proceeding on to the next unit. 

The strong form of the minimal comprehension principle suggests that 
the letter and word identification processes mentioned above merely set the 
stage for the act of comprehension, and that beyond these there is another 
sort of interaction between linguistic input and prior knowledge which must 
occur before the learner forms a stable representation of what is being read 
This additional form of interaction operates on larger "units" than previ- 
ously discussed' (though activities like word identification and word mean- 
ing may be affected) and is responsible for the "click of comprehension." 
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One way to _^hijTk about the d i fference between a strong and weak form 
of the minimal comprehension principle is to. distinguish between a reader's 
dictionary and a reader's encyclopedia as Clark and Clark (1977) have done. 
They suggest that a reader's dictionary consists of a menta 1 -s torehouse of 
information about v/ords, with each word in -a reader's vocabulary having 
three entries: the pronunciation of the word,, its syntactic category, and 
the meaning of the word. In comparison, the encyclopedia contains all of 
the individual's vyorld knowledge as it relates to words. So, for example, 
one*s encyclopedia entry for the^word dog might contain information about 
appearance, function, typical behavior, origins, history, experiential facts, 
etc. (Clark s Clark, 1977, p. ^11) * 

Given this distinction, the difference between a strong and a weak form 
of the minimal comprehension principle can be conceptualized as follows. 
The vyeak form of the minimal comprehension principle suggests that the reader 
looks up the meaning for each word in the dictionary and then strings the 
meaning of the individual words together to form the "comprehended" message. 
In contrast, the strong form of the minimal comprehension principle suggests 
that the reader consults both the encyclopedia and the dictionary. This 
means that the mental representation for a comprehended message would con- 
tain more information than was contained in the message itself. That is, our 
world knowledge contributes to and elaborates the incoming linguistic mes- 
sage. 

Our preference is for the strong version'^of the minimal .comprehension 
principle and much of the remaining section will be devoted to developing sup- 
port for this preference. We will first argue that comprehension is a natural 
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extension of the perceptual process, and since the perceptual process is 
a constructive process, the comprehension process i's likely to be also. 
Later, we will cdnsider several ways in which v/orld knowledge may interact 
with an incoming linguistic message during the comprehension process. 

Perception and Comprehens ion as Constructive Processes 

Our knowledge of the physiology and biochemistry of the perceptual pro- 
cess, along with the results of a great many studies having to do with the 
recognition of partial figures, visual illusions, pattern recognition, etc., 
have led to the conclusion that visual perception \s)a constructive process 
(cf. Neisser, 1967). That is, the perceiver must take a raw pattern of elec- 
tTOxhenLixaJ^.ti'mulation-^der^i^ved^ f rom 'a-pat-tern-of—Mght,-'and- construct some- 
meaning from that pattern. This interpretation process could work only if 
there were some record of similar patterns experienced in the past, and there 
was some way of linking the incoming pattern of stimulation to the previously 
recorded (and presumably labeled) patterns. 

There would surely be little argument that some aspects of the reading 
process involve an interaction between incoming stimulation and prior knowl- 
edge. Letter and word identification, for example, must involve processes 

r 

in which particular patterns of stimulation are linked to records of past 
experiences . 

But v/e want to make a stronger claim for the minimal comprehension 
principle. We want to argue that the processes leading up to the identifica- 
tion of the meaning of a word simply set the stage for the act of comprehen- 
sion. After the stage has been set, there is a further interaction between 
the linguistic input and prior knowledge which results in stable comprehension. 
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The arguments and evidence required to fully support our claim about 

I of- 

the act of comprehension would require more space than we have available 
in this article. But we will present several arguments, and several of the 
lines of evidence,, which we feel give the flavor of a more comfilete and com- 
pelling argument (see, for example, Anderson & Ortony, 1975; Bransford & 
McCarrell, 197^; Spiro, 1977) supporting the strong version of the minimal 
comprehension principle. 

It is obvious that we can read text—that is, process the printed text 
up to the point of determi ning what the words mean— and still not comprehend 
it. Bransford apd McCarrell (1974) have presented some simple, yet striking, 
illustrations of thi^. Consider these three sentences from their recent 
article: The notes were sour because the seam split. The haystack was ijn- 
portant because the cloth ripped. The trip was not delayed because the bot- 
tle broke: Most of us, when reading these sentences for the first time, dp 
not experience the "click'* of comprehension. This is so even though We under 
stood each of the words. However, in the context of the words, bagpipe^; para 
chute, and ship launching the sentences become immediately understandable. 

We believe t^iat the Bransford and McCarrell (1974) sentences illustrate 
3 fairly common situation. When our respective mothers sit down and read our 
latest articles (as they do), or when we sit down and read a book on Eastern 
religions (as one of us has recently attempted), the letters are identified, 
and meaning is attached to the words, but the sense of comprehension is un- 
certa in. 

The examples mentioned above can be interpreted in the context of Clark 
and Clark's (1977) previously mentioned distinction between the dictionary 
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and the encyclopedia. It is possible to imagine an ins Lance wheie ine woidb 
in a sentence have been located in the dictionary, but comprehension is dif- 
ficult because encyclopedia entries cannot be located which allow an inter- 
pretation of the words in context. We believe that this is precisely what 
-happens when encountering the Bransford and McCarrell sentences for the first 
time. 

The importance of consulting the encyclopedia while reading is apparent 
when considering a problem like the interpretation of polysemy. How, for. 
example, do we comprehend each of the following sentences?: Bill ran for 
mayor last fall. Sally ran to the store. The brook ran down the valley. 
The clock ran down last night. As it happens, -run is an extraordinarily 
p6lyseiTK)us word (52 distinct meanings in our desk dictionary), "and it is dif- 
ficult to imagine that our understanding of each of thfe above sentences is 
dependent upon retrieving the particular meaning which would allow a sen- 
sible interpretation of the sentences. Instead, it must be the case'that 
the environmental context in which each of the sentences is heard, and the 
sentential context for the word run, interact with our v/orld knowledge to 

h 

construct an interpretation of the sentences. Anderson and Ortony (1975) 
have presented several elegant arguments, and some compelling empirical evi- 
dence supporting this point. 

As another example of the necessity of world knowledge in interpreting 
text, consider the frequency with which we encounter, and the eas6 with which 
we understand, figural speech. Similes such as, "man is like a computer,'* 
or, metaphors like, "encyclopedias are goldmines," are encountered and under- 
stood many times a day. Yet it would be impossible to understand these 
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sentences if the reader were dependent ^.oi^nT^i^dua 1 word meanings... Context 
as It Telates to wlDrld"kno^ledge is"^crrttcal a1so*r^ -Gons-ider -the-met-aphor ieal 
phrase, "sheath thy impatience," which means one thing in the context of a 
discussion between a man and his young protege, and quite another in the con- 
text of a discussion between two lovers. 

These considerations lead us to the conclusion that the processes lead- 
ing up to the determination of individual word meanings are necessary but 
not sufficient conditions for comprehension. In order for comprehension to 
occur, there must be an interaction between the linguistic string and the 
reader's world knowledge base. 

If it is the case that comprehension is a constructive process resulting 
from an interaction between prior knowledge and the incoming linguistic mes- 
sage, then it must also be the case that memory representations which result 
from the comprehension process will be influenced by both the incoming lin- 
guistic message and 'the particular knowledge possressed. Since no two indivi- 
duals have the same experiential history, there will be differences in both 
the quantity and quality of prior knowledge which is relevant to the inter- 
pretation of a given linguistic message. Thus, any two individuals will have 
a somevyhat different representation of the same event. We will refer to this 
phenomenon as representational variability since its effects are observed on 
the outcome (the memory representation) of comprehension and not on the com- 
prehension process £er se . The implications of representat ion-il variability 
will be considered in a later section of this paper. 
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A Framework for Viewing Comprehension Processes 

^ ^vJ-sual T-upresentaHon-of^ the via'wpbint we have" been arguing is pre- 
sented in Figure 1. We do rtot claim that this framework is a'Tormal model " 
of comprehen^'fon/^ft^r do we claim, as the figure might suggest, that compre- 
hension involVes a linear progression through a series of discrete stages. 
Our Figure merely identifies processes or activities which must' (solid lines) 
or^may (dotted lines) be operative in comprehension. Thus, we, believe that 
the comprehension process mus£ entail the reception of linguistic information, 
the decoding of information with the aid of input from perceptual prior knowl- 
edge, and an act of comprehension which involves an interaction between world 
knowledge and the decoded linguistic message (our minimal comprehension prin- 
ciple). Factors which may be operative (dotted lines) are that perceptual 
knowledge may influence the actual reception of information, that world knowl- 
edge and cognitive strategies may play a role in "decoding, and that the recep- 
tion, decoding, and comprehension stages are interactive. These latter factors 
are supported in several models of the comprehension process (e.g., Clark S 
Clark, 1977; Schank, 1972), and Rumelhart (1977) has presented compelling 
arguments that comprehension processes must be interactive at all levels. The 
reason that the view presented in Figure 1 falls short of a forma 1 model of 
comprehension, is that it fails to specify how the processes we have identi- 
fied actually operate and interact. Later in the paper we shall review sev- 
eral models which have more fully inquired into the nature of these processes. 

The second limitation of Figure 1 is that it fails to adequately portray 
the interactive nature of comprehension. The processes identified are not pre- 
sumed to operate sequentially or discretely. Linguistic decoding, for instance 
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can involve input from sense receptors, recently comprehended information, 
world knowledge, and perceptual knowledge and simultaneously be outputting 
inforrr.at-ion-to Gomprehension-^pr-ocesse^., our store oi^woTl_d_ knowled^e^^etc. 
Furthermore, several messages can be operative in the system simultaneously, 
(for example, when one message is being comprehended, another may be under- 
going linguistic decoding). 



Insert Fi.gure 1 about here 



In the sections which follow, we will review several types of models 
which have elaborated upon the nature of the processes identified in Figure 1. 
While the details of these models will differ, we believe that the basic pro- 



cesses proposed in these models are compatable with those portrayed in Figure 1 

How the Minimal Comprehension Principle Might Work 

In the previous section we claimed that prior knowledge played a direct 
role in the language comprehension process. In this section of the paper we 
will examine specific ways in which prior knowledge might ^rnfluence compre- 
hension. This examination will include a brief review of a number of theories 
of reading comprehension. ^ 

Language representation and the minimal comprehension pr inciple . If some- 
one reads or listens i;o a linguistic message and then a short time later is 
questioned about what had b'een read or heard, we would find that they, had ex- 
eel lent memory for the meaning of the message but relatively poor memory for 
its surface form (e.g., Sach\^, 1367). We also know that mul I i 1 i ngua 1 s appear 
to process different languages^x^from a common base, and that pictures and verbal 
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materials are frequently translated into a common representational form in 
memory (e.g., Loftus, 1975; Loftus, Miller & Burns, 1978; Pezdek, 1977). 
And finally, weJ<now that when a message is comprehended, the prior knowl- 
edge which is relevant to TrvEerpreTing-the-message becomes part of the re- 
presentation of that message. For example, the re'p^resentation of the sen- 
tence, like appl es ." (Schank, 1972) will include the notion that the 
speaker is expressing an eat^*ng preference while such information would not 
be a part of the representation for the sentence, "I like Ike." Thus, it 
seems obvious that one part o^ the comprehension process involves inferences 
which are not part of an experienced linguistic message. 

All of the above facts point to the conclusion that there is not a one- 
to-one correspondence between language as it is experienced and language as 
it is represented in memory. A number of theorists of language comprehen- 
sion have acknowledged this conclusion and have attempted to develop a for- 
mal theory of the process v;hereby language is transformed and added to during 
comprehension. In this section we will examine a number of these represen- 
tation systems and the ways in vjhich they ar,e specifically influenced by prior 
knowledge. 

Kintsch*s theory . Walter Kintsch (197^) (Kintsch& Vipond, in press) h^s 
p^roposed one of the most cpmprehens i ve and explicit theories of language com- 
prehension. Kintsch proposes that the meaning of text can ,be represented in 
the form of a text base , which is a structured list of propositions. Pro- 
positions consist of a predicate with one or more arguments , and an argument 
is a concept, or a proposition itself. Concepts are realized at the language 
level by a word (or words ^if there are synonyms) or at times by a phrase. In 

12 
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essence, propositions are idea units representing a single idea. As Kintsch 
(197^) has written, "It is suggested here that propositions repre*^ *q 
and that language (or imagery) expresses propositions, and hence ideas. 
Thinking occurs at the proposi tiona 1 level; language is the expression of 
thought" (pg. 5). 

As an example of Kintsch' s representational system, the sentence, "A 
great black and yellow V-2 rocket forty-six feet long stood in the New Mexico 
desert," woufd be represented as: (l) (GREAT, ROCKET) , (2) (BLACK, ROCKET), 
(3) (YELLOW, ROCKET), [k) (y-2, ROCKEf), (5) (LONG, ROCKET), (6) (FORTY-SIX 
FEET, 5), (7) (STAND, ROCKET), (8) (IN, 7, DESERT), (9) (NEW MEXICO, DESERT). 
This example illustrates several of the conventions used in 'ihe representa- 
tion system. The name$ of concepts, as distinguished from words, are writ- 
ten in capital letters, and the predicates in the propositions are written 
first. In addition, the propositions are numbered such that when one pro- 
position serves as an argument for another (as in six and ei'ght above), the 
number of the proposition is written rather than writing out the proposition 
in i ts enti rety. 

'^Another aspect of Kintsch* s representational system Is text cohesion . 
Text cohesion is a concept which captures the difference between a list of 
numbered propositions derived from a written text, and a list of numbered 
propositions derived from random phrases. Kintsch suggests that the dif- 
ference lies in the degcee of argument repetition. As an example, note that 

i 

•eight of the nine propositions in the sample sentence show the argument 
ROCKET. Thus', an index of the degree of coherence present in a text is the 
extent to whi ch^ arguments are repeated across propositions. 

13 
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Kintsch has presented evidence to the effect that coherence in text is 
directly related to the ease of understanding of the text. Thus, a text 
which is highly coherent (having much argument repetition) will be easier 
to understand than one which is less coherent, in essence, what Kintsch is 
saying-is that new information in text will be understood more easily if it 
has recently been preceded by related information. This principle can be 
extended to extratextual information. Text will be understood more easily 
if the incoming arguments can be related to knowledge already in memory. 
The reasons for this facilitation become clear upon examining the process i ng 
aspects of Kintsch's theory. The ihterestoJ reader should examine Kintsch 
(1974) for the details of Kintsch's processing description. 

Kintsch's theory contains a number of instances of the critical role of 
prior knowledge.in constructing a memory representation for a linguistic 
event. First, Kintsch proposes a pattern matching phase in which^an incoming 
linguistic message (consisting of perceptual elements) is matched to a seman- 
tic me.TK>ry trace which consists of phonemic, graphemic , 'syntact i c, semantic, 
and experiential features associated with a given word. The perceptual- and 
semantic elements then combine to form an encoding for the event. Thus,' prior 
knowledge, in the form of semantic memory elements,Ms involved very early 
in the comprehension process. 

Prior knowledge also influences memory representation in Kintsch's theory 
through the role of short-term memory. Propositions which are already in 
short-term memory are presumed to influence both the pattern matching phase 
and the .encoding phase of the representational process. If needed, proposi- 
tions can also be called from long-term memory to aid in text interpretation. 
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This would occur, for example, when the prx^pos^tions presently in short-term 
memory were not relevant to the interpretation of incoming propositions. 

Thus, it seems obvious that the minimal comprehension principle is oper- 
ative at a number of stages in Kintsch's theory. In a later section of the 
paper we will consider what might happen when a reader does not possess the 
knowledge needed to interpret a text. This is what Kintsch and Vipond (in 
press) have to say about this situation: 

Another way in which knowledge would be beneficial --i n fact crucial — in 
comprehension is in the inference processes that are required whenever 
ah incoherent text base i s constructed . We have suggested that these 
Inferences constitute a major source of reading difficulty. For high- 
knowledge readers this difficulty should be greatly reduced, whereas 

for readers without the necessary knowledge it.,.would be i-risjurmountable 

\ 

and Jead to the formation of disjointed, impossi^ble to retrieve text 
bases, (p. 232) 

Schank's theory . A second formal theorv of language representation has 
been offered by Roger Schank (1972). The basic unit in Schank'*s theory is 
the concept, of which there are three kinds. A concept can be a nominal, 
an action, or a modifier. A nominal is something which can be thought of by 
itself, wi'^thout the necessity of relating it to some other concept. Concrete 
objects are the clearest example of nominals, and in fact, Schank refers to 
nominals as things which tend to produce pictures in the mind and abbreviates 
them as PP* s ^j(picture producers). 

An act ion--which Schank abbreviates ACT--is something that a nominal 
does. In general, actions take the form of verbs at the language level. 



Comprehension Theory and Measurement 
15 

A modifier is a concept which means nothin^g without the nominal or 
action to which it relates. The purpose of the modifier is to specify an 
attribute of .a nominal or an action. Schank refers to two types of modifiers: 
those which modify nomihals (which he refers to as PA's, for picture-aiders) 
and those which modify actions (labeled AA's, for action-aiders). In general, 
modifiers take the form of adjectives and adverbs at the language level. 

In addition to specifying categories of concepts, Schank also specifies 
the way in which the categories can relate to one another through what he 
calls "dependencies.** A dependency relationship between two conceptual items 
indicates that a dependent Item requires the presence of a governing item, 
but the conv.erse is not true. That is, a dependent must have a governor, but 
a governor need not have a dependent. ^ Using Schank 's terminology, PR's and 
ACT^s are inherently governing categories, whereas PA*s and AA's, are inhere- 
ently dependents. ' 

Using the basic categories mentipned above, and the notion of dependen- 
cies (greatly simplified here) as a means of linking concepts to one another, 
Schank constructs what he' calls "conceptual dependency networks." In essence, 
a conceptual dependency network is a representational scKeme which captures 
the conceptual elements of a language string, and the manner in which those 
elements relate to one another. 

Having developed a conceptually based representation system, Schank then 
specifies the processi'ng aspects of his theory. Language processing is divided 
into five stages or processors: (l) a syntactic processor which performs a 
preliminary syntactic analysis of a linguistic string; this analysis includes 
activities c»uch as finding main nouns and verbs in sentences, separating words 
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into their syntactic categories, and predicting the syntactic category of 
an upcomiag word based on the syntactic analysis of the current word; (2) a 
conceptual processor which separates words into their conceptual categories 
and identifies the dependency relationships that exist between the categories; 
(3) a conceptualization-memory interface which among other things relates two 
concepts or conceptual structures based on the individual experiences of the 
hearer, finds the funct ions of PP*s in the real world, and searches through 
conceptualizations for information to be stored in long-term memory; {k) inter- 
mediate memory whose function is to begin responses to incoming conceptual- 
izations and to interact with long-term memory; and (5) long term memory which 
is the* repository of information about particular subjects and experiences. 

It should be noted tha.t Schank's view of the activities of the. above 
processors is interactive rather than sequential. In his words, each of the 
processing stages "talk together while they work.*' This means that the activ- 
ities of one of the more basic processors (e.g., the syntactic processor) can 
be affected by higher order processors (e.g., intermediate memory) and vice- 
versa. 

In Schank's theory the involvement of the minimal comprehension prin- 
ciple* begins to explicitly come into play in thr conceptualization-memory 
interface where memory is consulted to determine the function of a given PP 
in the real world. The minimal comprehension principle is even more apparent 
in the activities of intermediate memory. There information from long-term 
memory is combined with input conceptualizations so that an interpretation 
process can occur. 



17 



Comprehension Theory and Measurement 
17 

Schank's interpretation process'^ti 1 izes the conceptual dependency net- 
works in conjunction with overall memory structure to establish what the lin- 
guistic string "really means.'* As an instance, he cites the case of someone 
coming into your office and saying, "Fire." It is obvious that the appro- 
priate interpretation of this utterance requires reliance on prior knowledge. 
In a simi lar, fashion, we must bring our prior knowledge to bear' in order to 
conclude that the sentence, "The man -1 i t a thoughtful cigarette," Is acceptable 
while the sentence, "The man lit a thoughtful fireplace" is anomalous (Lakoff, 
Note 1). And i f prior knowledge does not contain the appropriate information, 
it wi M be difficult to establish a stable interpretation of the message. 

Schema theories and the minimal comprehension principle . Another view 
cf how prior knowledge influences comprehension is contained in schema based 
theories. Rumelhart and Ortony (1977) and Schank and Abelson (1977) (they use 
script rather than schema) have written on how schemata influence the under-, 
standing process in general , and Anderson (in press) and Adams and Col linsdn 
press) have presented views oq how schemata specifically affect the reading 
comprehension process. ' ^ ' 

A schema can be conceptualized as a generic data structure which con- 
tains "slots" or place-holders for frequently experienced events. So, for 
instance, we could have a schema for re^cipes which would contain slots for 
ingredients, preparation instructions (mixing, etc.), and cooking instructions. 
Another example is the restaurant schema described by Schank and Abelson (1977) 
Consider these sentences and the question which fol^yvs as an instance of the 
use of a restaurant schema (Abelson, Note 2),: John went to a seafood restau- 
rant* for dinner. He ordered lobster. He paid with his credit card. What did 
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John have to eat? The answer to the question is so obvious that one has a 
sense of amazement upon realizing that the story says nothing about what 
John had to eat. It seems apparent that our prior knowledge about restau- 
rants set up some expectations about the message being comprehended. In a 
sense, our schema for restaurants added considerably to th.e content of the 
message which was read. 

Schemata can be said to exert their influence on comprehension in two 
principle ways: (l) they facilitate the memory representation process, and 
(2) they fill in missing content essential for complete understanding. When 
y^ie experience the same event repeatedly--whether that event is a sequence of 
actions as in the recipe schema or a more abstract, event si^ch as reading a 
journal article— we es tab lYs^h schemata for recording the essential content of 
those events. When the evenjt is encountered representation of the event \n 
memory occurs .easily because there is a pre-established structure for re- 
cording the content of the eyent. In contrast, when information is encoun- 
tered for which there is not a pre-established data structure, representa- 
tion (and comprehension) is difficult and laborious. In this case a data 
structure must be fabricated from what may be ill-suited existing structures. 
Events recorded in such a. way are likely to be forgotten quickly. 

The second way that schemata infl uence comprehension is by filling in mis- 
sing data slots. Even an unsophisticated analysis of the language we expe- 
rience while reading or listening reveals that an extraord ianry amount of in- 
ferencing must occur during the language comprehension process. When the com- 
munications we receive are about familiar topics schemata automatical ly compute 
values for content which may have been left out, but which is essential for 
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understanding. In contrast, when we do not have a schema for interpreting 
text content we have to consciously reason out what the missing content might 
be.-^ This process is slow, laborious, and not always successful. 

Linguistic conventions and the minimal comprehension principle . In this 
section we will consider several ways linguists and psycholinguists see prior 
knowledge influencing the comprehension process. These ways fall under the 
rubric of what are caMed linguistic conventions; that is^ unstated agree- 
ments as to the form, purpose, and function of linguistic communications. 

Herbert Clark and Eve Clark (e.g., I977) have written extens i vel y about 
what they call the^ g i ven-new contract . They suggest that every assertion made 
by a speaker or writer can be broken into two parts which can be distinguished 
by structural features of sentence: given information already known to the ' 
listener, ancj new information which the speaker believes to be true but is not 
known to the listener. Further, they suggest that there is an unstated "con- 
tract'* between speaker and hearer to construct sentences within the given-new 
framework. When a listener (or reader) hears an assertion they divide a sen- 
tence into its constituent elements, search memory for the given part of the 
-.assertion, and add the new information to it. 

The role of prior knowledge in the above description is obviously crit- 
ical iV one accepts the generalization that assertions always contain given 
and new information. If the given information is not in the comprehender ' s 
memory, i t wi 1 1 be difficult to understand and represent in memory the new 
information. 

A distinction similar to the given-new contract has been made by Halliday 
(1970) in his discussion of the thematic content of 1 inguistic messages. 
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Halliday suggests that a speaker *s (or writer's) judgement about a listener's 
current mental state is reflected in what is called thematic structure . Ir 
a sense, thematic structure reflects a constantly updated assessment of what 
a listener knows at any given point in time. Greatly simplified, the thematic 
content of a linguistic string is a concept which captures the notion of new 
information the speaker wants to convey combined with the^ Speaker * s assess- 
ment of what the listener already knows. The importance of prior knowledge 
•in this concept is obvious. 

. A third linguistic convention is the notion of speech acts (Searle, 1975). 
Searle suggests that all linguistic utterances can be classified into five 
categories of speech acts: (l) Representat ives— where the speaker Is convey- 
ing the belief that some proposi t ion is true, (2) Di rectives— where the speaker 
is trying to get the listener to do something, (3) Commi ssi ves--wbere the 
speaker is commi tting himself to some future course of action, {k) Express ives-- 
.where the speaker wishes to convey something about his **psycholog ica 1 state," 
and (5) Declarati ves--where the speaker' produces some new state of affairs as 
a function of the utterance (e.g., You're firedl). 

Jhe intent of Sear'le's (1975) classification system is to formalize the 
idea that what one understar^ds a message to >mean will depend in part on what 
one perceives the purpose /of the message to be^;. This is another instance of 
the way in which priory knowledge influences the comprehension process. 

The final lingu is/tic convention to be discussed here is the notion of 
types of discourse (e.g., Cunningham, 1978). This view holds that humans have 
developed particular stylized ways of communicating information via discourse 
including such modes as narratives, exposition, description, etc. Each of these 
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types is presumed to have particular structural characteristics associated 
with it. For example, Rumelhart (197^) has proposed a grammar which he be- 
lieves describes the .structura 1 characteristics of well formed simple stories. 

It IS further assume^a that through exposure to thes^ various forms of dis- 

/ 

course, humans develop cognitive structures which correspond to the struc- 
tural characteristics of the discourse type. These cognitive structures gov- 
ern the set of expectations that readers have about the type of information 
which should occur in a given text. These expectations in turn influence How 
incoming text information is processed and remembered. 

To date, the majority of research on types of discourse has dealt with a 
single type of text--the simple story. For example, Thorndyke (1977), build- 
ing on the work of Rumelhart (197't), has attempted to document the validity 
of this grammar of stories, conceived as a set of rules by means of which such 
stories are structured. A story is conceived as consisting of a setting, theme, 
plot, and resolution-i -each of which is further decomposed into' subcategories. 
The validity of thjs model was tested by inspecting the pattern of subject re- 
call of the stories with respect to the categories proposed and the relations 
among them. Predictable differences in recall were found as a function of - 
these categories in "ideal" stories and in stories in which some of the cate- 
gories were deleted or rearranged. Stein and Glenn (1977) have been proceeding 
independently along similar lines but have, in addition, identified interest- 
ing developmental differences in children's ability to comprehend various struc- 
tural features of stories. Knowledge of the conventions employed in various 
discourse is thus another example of the importance of prior knowledge for 
comprehension. 
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In summary, each of the positions we have reviewed agrees that prior 
knowledge plays a critical role in the comprehension process. The theorists 
emphasize different types of knowledge crucial to comprehension, but they 
would all agree that the minimal comprehension principle is an essential part 
of any theoretical effort. 

Implications of the Minimal Comprehension 
Principle for the Measurement of 
Reading Comprehension 

Thus far in the paper we have claimed that^prior knowledge has a direct 
and critical role in the language comprehension process, and we have suggested 
several waysin which prior knowledge could influence comprehension. In this 
section of the paper we wi I I trace the implications of the minimal comprehen- 
sion principle for the measurement of reading comprehension. 

If one accepts the plausibility of the strong version of the minimal 
comprehension principle, there are two classes of reasons why a reader might 
not satisfactori ly comprehend a segment of written text. The first class of 
reasons is that the reader might fail to fulfill the lower level requirements 
for text comprehension. That is, due to poor basic skills such as the decoding 
of words and word- identi f ication, or the inability to combine individual words 
into coherent messages, the reader fails to comprehend ;he text. The second 
class of reas6ns for poor comprehension is that the reader may lack the world 

knowledge necessary to establish a stable interpretation of the linguistic mes- 

ft 

sage. This could result in the inability to construct an interpretation of 
the message, or it could result in an interpretation that was so unstable or 
deviant (e.g., implausible) that it disrupted subsequent comprehension. In 
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either case, it could be concluded that the reader did not grasp the intended 
understanding of the text. 

In the paragraphs which follow we will cl'aim that most existing tests 
of reading comprehension cannot distinguish between the two classes of -reasons 
mentioned above, and that this restricts the utility of the tests to situa- 
tions where one wants an instrument to predict future comprehension perfor- 
mance, rather than assessing or diagnosing current' comprehens ion performance. 
Further, we will suggest that the inability of the tests to distinguish be- 
tween the two classes of reasons for poor performance may have the consequence 
of underestimating the reading gains achieved by national reading programs. 

Question Type, Pas sa g e Content, and the Relation Between Reading Comprehension 
Tests and Ability Tests 

The typical reading comprehension test asks the reader to read a segment 
of text, and then, either during (in the case of cloze tests) or after reading, 
the reader selects one of four alternatives as a response to a question or 
blank. We want to examine both the type of questions asked and the content of 
the. text sample in light of the minimal comprehension principle. 

There are two kinds of questions readers respond to on reading compre- 
hension tests. The -first kind of question can be answered solely on the basis 
of the passage coQtent the student has just read. As is traditional, we wi 1 1 
call this type of que^i^^n a literal comprehension question. Other questions 
(or blanks to be filled ih) require that- the reader draw on information that 
has just been read, and qn information acquired previously, in order to respond 
correctly. Again in accordance with tradition, we will call these inferential 



24 



Comprehension Theory and Measurement 
comprehension questions, since responding correctly to these questions re- 
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quires Chat >the reader draw together just read and previously known infor- 
mation. 

There are two important things to note about inferential comprehension 
questions. The first is that they require that the reader perform an, infer- 
ential reasoning activity. If the required reasoning activity^ is at all dif- 
ficult, it is likely that reasoning abi 1 i ty wi 1 1 play a part in determining 
reader performance. 

The second thing to notice about inferential questions is that successful 
performance is dependent on the reader having appropriate prior knowledge. 
Imagine, for example, that a passage in a reading comprehension test tells 
about a boy and his father going camping. Further, imagine that one of the 
questions asks the reader, "What did Tom's father probably use to cut the 
firewood?"' Obviously, the reader who has acquired information about camping — 
or better yet, gone camping— will do better th^n readers who do not have this 
information in memory. 

The' extent to which prior knowledge is shared between the test constructor 
and test taker will be important also. This point relates to the represen- 
tational variability issue raised in the introductory section of this paper. 
If a reader and a writer have quite different experiential histories, it is 
possible that a reader could construct an interpretation for a text message 
that is different from the one the author intended. The reader would, in this 
case, perform poorly on questions probing for the author's intended text inter- 
pretation, even though the reader had, constructed a perfectly sensible (based 
on his experiential history) text interpretation of his own. 
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The two examples above point to the conclusion that the broader the gen- 
eral knowledge base of a reader, and the greater the extent to which author 
and reader share a common knowledge base, the more likely it is that the 
reader will perform well on inferential questions based on the text passages. 

The fact that prior knowledge plays a part in answering inferential 
questions has been empirically documented. Jim Pichert (personal communication), 
for example, recently asked seventeen secretaries and staff members at the 
Center for the Study of Reading, Uni vers hty. of Illinois, to complete the test 
items on a commonly used reading comprehension test (about fifth grade level) 
without reading the passages. The average score was 33 correct out of a pos- 
sible 45. In addition, for 36 out of the ^5 items, the scores were signifi- 
cantly above chance. One might argue that Pichert' s demonstration is not that 
convincing since the tests were not designed for use with adult populations. 
However, both Pyrczak (1972) and Tujnman (1973-7^) have shown similar effects 
(though of lesser magnitudes) with subjects using age appropriate materials. 

TBe claim that reasoning ability and the extent and nature of prior 
knowledge influences performance on inferential test questions leads to a gen- 
eral conclusion: independent of reading skill, there will be a relationship 
between general ability (as indexed by ability tests) and performance on infer- 
ential comprehension questions. This conclusion follows given that one be- 
lieves that reasoning ability and extent of knowledge acquisition are both 
components of (or resultants of) general ability. 

Let us now turn to the issue of passage content in reading comprehension 
tests. This is largely speculation, but we believe that the content of reading 
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comprehension test passages can best be understood in the context of the 
cultural bias issue--with cultural bias being the extent to which test con- 
tent unfairly favors one cultural group over another. 

In general, developers of mental tests have been very sensitive to the 
issue of cultural bias in tests, and the developers of reading comprehension 
tests are no exceptions to this rule. One v/ay that a test developer can 
guard against task irrelevant bias creeping into a test is by keeping to a 
minimum the degree of cultural loading (cf. Jensen, 1976) in the test. Cul- 
tural loading is a term used to describe the degree of cultural generality 
or specificity ^or a given test item. Thus, the more narrow or less general 
the cxjlture in which the informational content of trie item could be acquired,^ 
the greater the degree of cultural loading for the item. So, for example, 
'*Name the largest city park in San Francisco," is a high cultural loading item, 
whereas, "Name the first president of the United States," is a low cultural 
loading item. 

The concept of cultural loading is applicable to reading comprehension 
tests even though the focus is not on test items. Instead, one can think of 
the topical content of test passages as being subject to cultural loaB'ing. So, 
for example, a passage about the ethnic diversity of New York City neighbor- 
hoods would have high cultural loading relative to a passage about taking an 
automobile trip to visit a relative. 

Our speculation is that developers of reading comprehension tests have 
either consciously, or perhaps intuitively, attempted to avoid the charge of 
culturally biased testi by utilizing reading passages containing content 
equally accessible (in cheory) to all members of the population. So for 
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example, one popular reading comprehension test contains successive passages 

about owning waterfront property, the behavior of hummingbirds, and" world 

/ 

class milers. Taken as a group, it is reasonable to assume th^t the infor- 
matlon about these topics is equally accessible to a 1 1 members of the popu- 
lation. However, equal accessibility does not mean equal acquisition. 

By definition, .the information required to interpret a low culturally 
loaded passage is not likely to be acquired by interacting with one's immediate 
environment.. Ins^.ead, the information must be acquired from sources conveying 
Information about the surrounding world. This means that acquisition of low 
cultural loading topical content (through reading, conversation, popular media, 
educational media, etc.) is likely to be related to variables like socioeconomic 
status, which in turn are strongly related to performance on aptitude tests. 
This assumes that socioeconomic level is likely to be related to the degree to 
which one takes advantage of sources of information about the world outside of 
one's immediate environment. \ 

The line of reasoning pursued above, when joined with the implications of 
the minimal comprehension principle, leads to the conclusion that those who 
take advantage of sources of knowledge about the surrounding v/orld are likely to 
be in the best position to interpret the content in reading comprehension test 
passages, and further, those who take advantage of knowledge acquisition oppor- 
tunities are also likely to be those who enjoy other advantages associate^ with 
higher economic status. 

In the previous paragraphs we have discussed the tyf5es of questions asked 
on reading comprehension tests, and the type of content contained in test pas- 
sages, and have concluded that each of these features contributes to some extent 
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to the relationship between reading comprehension test performance and abil- 
ity test performance. The important poi nt in this conclusion is that some * 
part of a reader's comprehension test^performance is independent of reading 
skill, per se . That is, when a student\performs well on a reading compre- 
hension test we can safely say that he has mastered appropriate reading skills. 
But what can we say when he performs poorl\y? Poor performance might be attKib- 
utable to lack of basic reading skills, or ft might be attributable to a lack 
of prior knowledge necessary to interpret the text. We have no way, from the 
test at least, of determining which of these is the case. 

In the next section we will trace the implications of this dilemma as they 
relate to the uses for reading comprehiens ion tests. 

The Uses for Reading Comprehension Tests 

9 

' Reading comprehension tests can be used for three general purposes which 
can be discussed from the perspective of the framework presented in Figure 1. 
The purposes are: (.1) to predict future comprehension performance; (2) to 
diagnose reading difficulties; and (3) to assess educational gain. Current 
tests of reading comprehension differ in the extent to which they can satisfy 
each of these purposes. 

One use for reading comprehension tests is, to predict future performance. 
This means that on the basis of a score obtained now, one woul^^ like to prer 
diet performance on a future occasion where both the time of test^ing and the 
nature of the reading materials are^ unspecified. A test which would be ideally 
suited for this purpose would be one which tapped both general reading skills, 
and range of prior knowledge; since these are two attributes Which are going 
to be critical in an unspecified future occasion. 
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Re'ferring to* Figure 1, a test designed for predictive purposes would 
be sensitive to a^l qf^ the activities depicted in the figure. That is, since 
the successful word identification act must involve the ability to receive 
and decode linguistic information, and since the comprehension act must in- 
volve an interaction betw en a decoded nnguistic message and world knowl- 
edge, a good predrctive instrument would be one which tapped abilities rel- 
evant to all of " these activities. ^ 

Many of the standardized reading comprehension test currently in use 
seem ideally suited for the purpose of predicting reading comprehension per- 
formance. Successful responding to the questions on a test, or to the blanks 
jn^*^ cloze test, assures that examinees possess basic decoding skills. In 
addition, the presence of low cultural loading text passages means that per- 
.formai^ce is going to be influenced by the extent of world knowledge. As an 
added feature, most standardized tests are sensitive to reasoning and infer- 
ential .abi 1 i ti es. This is particularly true of cloze based tests, and is 
true of Veading comprehension tests in general to the degree that one author 
has concluded that "if reading isn't reasoning, then maybe reasoning is read- 
ing" (Thorndike, 1973-7^)^ The inclusion of reasoning ability probably adds 
to the abi\lity of the test to predict performance on some future occasion 

since reasoning ability may, as Kintsch and Vipond (in press) have suggested/, 

/ 

be very important in interpreting text having an incoherent text base. / 
The evaluation that current comprehension tests seem ideally suited^,,for 
predictive purposes does not mean that we believe they are ideal predictive 
instruments. Rather, we believe that the tests lend weight to those factors, 
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illustrated in Figure 1, which are going to be important in comprehending^ 
an unspecified text sample at a future point in time. 

Whereas we are positive about the use of current comprehension tests 
for predictive purposes, we are less sanguine about their use for diagnostic 
purposes. The problem is that one cannot tell whether poor perfoV>(ria0^e means 
that the student is deficient in reading skills, or whether it means that he 
does not ^i^ave the prior knowledge needed to construct an interpretation of 
a text passage. Given the inability to distinguish between these two classes 
of reasons for failure, the test has minimal utility as a diagnostic instru- 
ment. 

The problem can be illustrated by reference to Figure 1. When diagnosing 
reading difficulties the most basic decision that needs to be made is that a 
difficulty exists. But in tests where weight is given to extent of prior knowl- 
edge one can never tell whether poor performance is due to lack of some crit- 
ical reading skill, lack of world knowledge needed to interpret the text, or 
to some failure in the process whereby a decoded message interacts with appro- 
priate prior knowledge. This suggests the possibility that failure could occur- 
due to lack of relevant world know I edge-when the student had perfectly good 
reading comprehension skills. ' 

Many tests of reading comprehension contain item type features which 
presumably are for diagnostic purposes. So. for example, one can get score's, 
for identification of main ideas, sentence meaning, recall of facts, inferred 
meaning and character analysis, and authors attitude and techniques of per- 
suasion. Aside from the world knowledge problem there is some question as to 
whether these item types have diagnostic utility. The first difficulty is that 
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the subscores are generally based on a small number.of. items which results 
in considerable instability in the scores. But more important is the fact 
that the subscores are rarely keyed to a theory or mode'l of the comprehension 
process, instead the scores generally are named after skills thought to be 
important \n reading instruction, or after factor labels derived from factor 
analyses of reading comprehension tests. As yet, there is no research which 
demonstrates that there is a relationship between knowledge of the sort pro- 
vided by comprehension test subscores and improvement in instructional de- 
cision making. 

Most reading comprehension tests are also not well suited for the pur- 
pose of assessing educational gain. Since this is probably the most frequent 
use for the tests— and certainly, from an educational policy point of view, 
the most critical--we will develop our position, and its ramifications, with 
care. 

There are .several reasons why most reading comprehension tests are ill- 
suited for the purpose of assessing educational gain. Among these are score 
interpretation procedures which emphasize comparisons between individuals 
rather than within individuals, and item selection techniques which produce 
tests with good'predicti ve properties, but relatively poor assessment pro- 
perties (see Carver, 1974, for an excellent discussion of these). Our focus, 
however, will be on the property we discussed preyiously: the inabili'ty to 
determine whether poor performance is due to popr reading skills or to lack 
of critical prior knowledge. 

A reading comprehension test used for the purpose of assessing educational 
gain should be able to identify the extent to which a text sample is comprehended, 
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and it should document comprehension gains which occur from one tes^t occas ion^ 

to the next, This means that the test should be sensitive to readj^g^^ki 1 1 

gains, b^u.t- it should not be sensitive to world knowledg^sTns . Our previous 

discussion of the implications of the mini maj^omprehens ion pri nci pi e j nd i cated 



that most /tests of reading compreheps-f-^rrdo not satisfy these requirements 



Let's take a concrete^Kampl e . Assume that a student at the beginning of 
a school year achj.eVe's a raw score of kS (we'll ignore the problem of what 
standard scares mean). After a year of remedial instruction the student is 
tested again, and again scores kS- Does this mean that the instruction has 
failed (ignoring the possibility of measurement error)? The problem is that 
we can't tell. The student may have improved considerably in reading skills, 
but lacks fhe knowledge ba,se needed to construct the intended interpretation of 
the test passages.- Notice that this problem occurs in the case of gains also. 
Assume that our student who scored kS at the beginning, of the school year scores 
60 at the end of the year. Does the gain reflect an increase in skills as a 
function of instruction, does i t reflect an increase in the student-'^s-knov7redge^*^^ 
base, or does it reflect some combination of both of these? / 

While the examples above may^e extreme, they illustrate the dilemrri^ as- 
sociated with using current comprehension tests to document educational/ gai n. 
Considering the extent to which reading comprehension test scores have^'con- 
tributed to educational policy over the past ten to fifteen years, the impli- 
cations of this analysis are profound. We are suggesting that most reading /V 

//\ 

comprehension tests can be insensitive to gains in reading skills, and moreover ^ 
that the tests are most likely to be insens^ftive to gains in those very groups 
where the documentation of gain would be most important. 
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The conclusion above follows from, our previous argument about the re- 
lationship between socioeconomic status and performance, on reading compre- 
hension tests. We argued tfjat low socioeconomic groups were likely to have 
a world knowledge deficit relative to more advantaged groups because of a 
lower likelihood of taking advantage of sources of world knowledge. This 
deficit would result in an increased probability that a student would per- 
form poorly on the test because of lack of knowledge needed to interpret the 
text. And this, in turn, would result in a relative insensi ti vi ty of the 
test^tp actual educational gain. And, of course, it is true that the bulk 
of the remedial reading effort in this country is targeted at the lower 
socioeconomic segment of our society (Title 1 of the ESEA, for example, dic- 
tates this). 

It is a fact that in the last fifteen years programs designed to improve 

reading skills in disadvantaged populations havejia^^ 

when those^re^uj^tsjia^^ standardized reading comprehension 

test. We are suggesting that at least part of these disappointing results 
could be attributable to an i nsens i t i vi ty of the tests to instructional gain. 
This leaves open the possibility that true instructional gain occurred. 

Recommendations for the Measurement 
of Reading Comprehension 
In the previous section jt was argued that most reading comprehension 
tests were not ideally suited for the purposes of diagnosing reading problems 
and assessing educational gain, but the tests were better suited for the pur- 
pose of predicting future comprehension performance. In this section we will 
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discuss ways "in which comprehension tests could be constructed so as to im- " 
prove their diagnostic and gain assessment utility. 

According to our previous analyses, most test of reading comprehension 
are not good measures for diagnostic and evaluative purposes because one can- 
not distinguish between poor performance due to/lack of reading skills and 
poor performance due to lack of critical world knowledge. In the paragraphs 
which follow it^will be argued that a more suitable way to assess comprehen- 
sion for evaluative and diagnostic purposes is to use techniques which are 
les-s sensitive to the influences of reasoning ability and prior knowledge. 
It will also be argued that an ideal diagnostic test must be based on a devel- 
opmental theory of reading comprehension. 

Before conside ring thes e^argi^ issues 
which^shouid-beTiacJe explicit. Like Carroll (1977), we subscribe to the view 
that reading comprehension is a special case of language comprehension and 
that the extent of reading comprehension will be limited by the extent of 
language comprehension. This means that, in general, one vyould not be able 
to read and understand something that one could not listen and comprehend-- 
given that the J ister^fng and reading situations were comparable. 

We also subsc/ibe to .the view that .comprehension entails a process where- 
by an incoming linguistic message interacts with the world knowledge base, and 
is transformed into a representation which preserves the meaning of the mes- 
5=»ge, but not its form (Anderson, 1972; Sachs, 1967). This act of comprehen- 
sion can be contrasted with other learning acts, such as rote memorization, 
where the form of the message may be preserved, but not its meaning. These 
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acts, such as the meaningless memorization of a foreign language phrase, 
would onot, in our terms, ^involve comprehension. 

Another assumption is that a comprehended message is represented in 
memory in an enduring fashion. There are issues involved in this assumption 
which are controversial. For example, John Carroll has written: 

If comprehension is a process that ocr-urs more or less simultaneously 
with the reception of a message, we would be interesteo in the occur- 
rence or nonoccurrence of that process only during the reception of 
the message or<at least within a very short tAnii^^ 

'S to be invoTved^^a^ has been called short- 

term memory, i.e., memory that can fade within a few seconds. As soon 
as longer t ime-intervals are involved in the testing of comprehension, 
there is the possibility that we are studying memory processes along 
v/ith, or in place of, comprehension processess. (1972, p. 6) 
Carroll's position severely restricts the ways in which comprehension 
might be measured. One can either ask the examinee to respond to a message 
within a matter of seconds after receiving it,, or one can ask the individual 
to respond to a message which can be reexamined at will. 

There are a number of problems with Carroll's position. One of these is 
the fact that there is more evidence for individual differences in short-term 
memory (e.g., Hunt, 1977; Hunt, Frost, & Lunneborg, 1973; Hunt, Lunneborg, & 
Lewis, 1975) than there is for individual differences in long-term memory 
(e.g., Shuell S Keppei, 1970; Underwood, 195^; however, see Royer, Hambleton, 
S Cadorette, 1978, for a dissenting view), thereby raising a question as to 
whether memory effects can ever be eliminated, or even reduced. Another 
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problem Is that there are inherent limitations in the two kinds of tasks 
'(short term responding and , free examlna-tion of the message) which meet 
Carroll's requirements. The short term responding task works ideally, only 
in the situation where a behavioral response is supposed to follow a request, 
and it is difficult to generate sensible verbal questIons--other than,4nfer- " ^ 
ential quest ions— when the examinee can _examfnB''tRe'^ssage at will. 

^Our^^gosXtion-TS'THa^^ processes and memory processes are 

inextricably intertwined. In fact, our defln.ition of a comprehended message 
IS In terms of the form of the message representation in memory. Further, we 
assume that a comprehended messaSge will be retained in memory better than an 
uncomprehended message. There is ample evidence to support this assumption 
(e.g., Bransford & Johnson, 1972; Bransfbrd S Johnson, 1973; Pooling & Lachman , 
'1970- - ' * 

Having made our assumptions explicit, we are now ready to consider how- 
to reduce the effects of prior knowledge and reasoning ability in comprehen- 
sion testing, when the tests are to be used for evaluative purposes. The 
variability in comprehension test scores attributable to differences in prior 
knowledge could be reduced by matching topical content of the passages to the 
knowledge background of the examinees. In essence, what we are proposing Is 
a kind of "tailored-testing" technique based on matching prior knowledge to 
topical content. One can imagine a sort of "passage- bank" which would be akin 
to item banks which are now available for preparing criterion referenced tests. 
Teachers and administrators could then select passages from this bartk which 
would be matched to the prior knowledge of a group of students, or even per- 
haps, to the prior knowledge of individual students. 
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The obvious question about the above proposal is how to determine which 
passages should be matched to which examinees. lh\s is a questions which -^.^ 
should be -researched, but we do have several ideas about it. First, it may 
not be an enormous problem. Our guess is that teachers and administrators 
are very good judges of whether topical material will be in the knowledge 
repertoireof students in their charge. If this v^ere not true there are ways 
in which the question could be approached objectively. 

One way to objectively- approach the topical content/prior knowledge 
match is to conceptualize prior knowledge as sets of schemata (cf. Anderson, 
in press; Adams S Collins, in press). Anderson (in press), for example, 
views schemata as structures for the representation of generic knowledge. 
Each schema is imagined to contain a slot for each bit of generic knowledge. 
Thought of this way, it is apparent that it is not specific information which 
will be important in the acquisition of related information. Rather, it is 
the extent to which the individual has acquired generic information related 
to the topic. This suggests the possibility that techniques designed to 
assess generic information would provide a good index of the extent to which 
a reader possesses prior knowledge needed to interpret text on a given topic. 
One way this might be done is to ask the individual to identify items which 
are generically true for a given topic (e.g. , what is true of every baseball 
game you can imagine; or what is true of every cow that exists) when presented 
with a list which contained both generically true items, and items which v^ere 
not generically true. Presumably, performance on this task would be /elated 
to the ease with which the individual could interpret th^ topical content of 
a passage. ^ 
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The second concern in comprehension measurement is how to reduce the- 

contribution of reasoning abil'ity to test performance. Our proposals are 

based on a concept of how a ^comprehended message "is represented in memory. 

We previously indicated that our concept of comprehension was the process 

whereby a linguistic message is translated into a representation which pre- 

♦ 

serves the, meaning of the message but not its surface structure. Thought ' 
of this way, one can distinguish between, tests which assess the form of a 
memory representation versus tests which measure operations (cognitive 
manipulations) on that form. When used for evaluative purposes, the ideal 
comprehension test would be sensitive to the form of the memory repre^enta- 
tion, but insensitive to operations on that representation. The extent to 
which the test is sensitive to operations on the representation is the ex- 
tent to which the test is likely to be measuring general ability factors in 
addition to reading comprehension, per se . 

There are undoubtedly a number of ways of assessing the form of a mem- 
ory representation without assessing operations on that representation. V/e 
will talk about three which might satisfy this requirement. The first is a 
technique which involves presenting examinees with linguistic materials and 
then asking them to choose a picture, from several presented, which best re- 
presents the meaning of thj linguistic material. This procedure has been 
frequently used in foreign language comprehension tests (Carroll, 1971), and 
has been recently used with appaj^ent success in a national assessment of 
reading skills conducted in. Australia (John Elkins, personal communication). 
This latter use was particularly interesting since the test samples were 
drawn from a mock newspaper. 
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^ The picture identification task i s* theoretical ly consistent with our 
requirements since there is now ample evidence that pictorial and verbal 
material are frequently translated into the same' representational form in 
memory (e.g., Loftus, 1975; Loftus, Miller, & Burns, 1978; Pezdek, 1977). 
This means that comprehension can be assessed by having the examinee match 
a representation derived from a picture, which presumably is easy to compre- 
hend, to one derived frohi a, verbal message.^ This matching process should 
it 

occur with minimal cognitive operations on the tvio representations. 

A second technique for measuring reading comprehension which may b*e 
"insensitive to reasonjng abilities is a variant of a sentence verification 
task (cf. Pezdek Fr Rdyer, 197^; Sachs, 1967) currently being investigated 
By J. Royer. This technique involves preparing four versions of each of 
the sentences in a text passage: (l) the sentence as it originally appeared; 
(2) a paraphrase of the original sentence; (3) a meaning change version of 
^the sentence which preserves as nearly as possible the origin^al-wording in 
the sentence; and (4) a distractor which is similar in length and complexity 
to an original sentence and is semantical ly consistent with the topical con- s 
tent of the text passage, but is not semahtically similar to any of the orig- 
inal sentences. An example of each of these types of sentences drawn from 
a story about trapping wolves is presented below: 

(1) But morning after morning as I rode forth to learn the result, 
I found that all my efforts had been useless, (original) 

(2) Blit day after day at early sunrise as, I went forth to discover 

the outcome, I learned that all my attempts had failed, (paraphrase) 
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(3) On nx)rning after moral ng as I rode forth to learn the result, 
T found that all my efforts had been successful. (mVani ng^^^^^^^^^ 
/ The cowboys and I traveled the length and breadth of the great 

mesa, but our prey always avoided us. (distractor) 
The sentences above are used In a task which Involves having the stu- 
dent read a twelve sentence segment of text, and then rating 16 test sen- 
tences (k original, k paraphrase, etc.) as being "old" or "new." The 
readers are instructed to rate. a sentence old if it is the same as or means 
-the same as an originally appearing sentence; otherwi se the sentence is to 
be rated new. After scoring, the data is submitted to signal detection 
analysis (e.g., Coombs, Dawes, & Tversky, 1570) to separate response bias 
from response accuracy.. ^ 

Pr^l ijninary data from s.tudies using Royer's technique appears prom- 
ising In that the technique seems to yield reliable results, is sensitive 
to varying^fficul ty In reading' materials, and is highly correlated to 
standardized reading comprehension test performance but is less highly cor- 
related with IQ measures than are standardized tests. This may mean that 
* the technique is less sensitive to variability In reasoning ability than 
are standardized tests. 

Another feature of the sentence verification procedure Is its possibil- 
ities as a diagnostic technique. For example, a reader who was overly re- 
liant on a word by word analysis of text might be expected to respond "old" 
to original sentences (correctly) and meaning change sentences (incorrectly) 
and to respond"new" to paraphrases (incorrectly) and distractors (correctly). 
, In a similar fashion, a reader who was overly reliant on higher .level' 
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processing— a "top-down'* processor, to use Rumelhart*s (1577) phrase-- 

. - ^ might respond.-'lo.l.d." .to -or-iginal- sentences, -paraphrases-r-and-distpa&tor-s-- 

(since all of these ar4 semantical ly consistent with the theme of the passage) 
and respond "new" to meaning changes (since these frequently violate the gist 
of the passage). Other patterns of responding might be signals for other 
. sorts of reading difficulties. 

A third technique for measuring reading comprehension which appears 
promising would involve both listening and reading (see Carroll, 1977, for 
arguments about comb'ined listening/reading tests). One possibility would 
be to read a sentence and then have the examinees select a sentence from 
among several alternatives (Royer's paraphrase, original, meaning change, 
and distraction sentences might provide items for a response set) which 
meant the same thing as the sentence which was heard, A variant of this 
procedure could be used to measure both listening and reading comprehension 
simultaneously. This could be done by having the student listen to a sen- 
tence, and then selecting both a picture and a sentence which meant the same 
thing as the sentirnce which was heard. 

The techniques described above have a number of shortcomings. For 
example, Carroll (1971) has mentioned several- difficulties with the picture 
matching procedure which would linit its-u^e. These include problems with 
guessing, the fact that only certain materials lend themselves to pictorial 
representation, difficulty in preparing Appropriate pictorial materials, 
and the fact that a picture cannot capture all the lexical and grammatical 
material that a sentence might contain. These difficulties vary In impor- 
• tance. The problem with only some material lending itself to pictorial 
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representation clearly limits the use of the technique to certain situations. 
Other problems, however, are not so important. On most occasions it would 
not be necessary to capture all of the grammatical and lexical nuances of a 
verbal message in a picture. In addition, .the guessing problem is critical 
only if one wants to assess the comprehension of a given verbaj message with 
only one or two test items. 

A possible limitation of Royer's technique, and for that matter, the 
other techniques under discussion, is that it is most applicable to singl'e 
sentences responding. It may be possible to expand the technique to more 
than single sentence units, but, thus far, such an attempt has not been 
made. 

The 1 istening/reading techniques suffer from the same limitations pre- 
viously mentioned for the picture and sentence verification tasks. That is, 
it IS not possible to develop pictures which are comparable in meaning to 
all types of text, and the techniques are probably limited to single sentence 
units. 

y 

The techniques for measuring comprehension discussed in the section 
above would have the most utility when the tests were being used for evalua- 
tive purposes. The techniques are likely to be less useful when the tests 
are to be used for diagnostic purposes. The ideal diagnostic instrument 
would be based on a developmental theory of the comprehension process. That 
is, one would like to know when certain processes and cognitive structures 
normatively appear, the sequence in which they appear, and the sorts of ex- 
periences which lead to their appearance. We are obviously a long way from 
having such a theory, but the beginnings of a theory seem to be emerging 
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from recent work on prose comprehension in children (see Stein, in press, 
for a review of this literature). 

In this section of the paper we have argued that a reading comprehen- 
sion test to be used for evaluative purposes would ideally be sensitive to 
the form of memory representation and would be Jnsens i tive to differences 
in prior knowledge and reasoning ability. We want. to make clear that we 
do not believe that prior knowledge and reasoning ability can be eliminated 
entirely as sources of variability in reading comprehension testing. For 
example, it is surely the case that extent of prior knowledge is going to 
influence the degree of comprehension: This becomes a factor in comprehen- 
sion testing since two individuals who have been judged to be familiar with 
the topical content of a passage may still vary in their extent of familiarity, 
and this in turn inay influence their test performance. 

In a similar fashion, it is likely to be the case that inferential pro- 
cessses are an automatic part of the reading process (Rumelhart, 1977; 
Schank S Abelson, 1977), and that it is nons^ensical to suggest totally remov- 
ing inferencing from comprehension testing. What one can do, however, is re- 
move items which require the conscious search of existing memory and the inte- 
gration of the results of that search with something which has recently been 
read. These sorts of items, surely lend weight to general ability factors,. - 
and detract from the measurement of reading comprehension,^ per se. " 

Summary and Conclusions . ' 

Several years ago Simmons (1971) surveyed the comprehension testing 
literature and identified seven appro^cjhes which have been employed in 
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defining and measuring reading comprehension (i .e. , ski 1 ]s, measurement, 
factor analytic, correlational, readability, introspective, and models). 
Of these approaches, \t is probably safe to say that the theoretical model 
approach has had the least practical impact. One purpose of this paper was 
to attempt to show that comprehension theory can make some contribution to 
the measurement of reading comprehension. ' ^ 

We have argued, that there is a theoretical principle which must be part 
of any serious theory of reading comprhens ion . This principle, which we 
called the minimal comprehension principle, asserts that the act of compre- 
hension must entail an interaction between an incoming linguistic message 
and the reader's world knowledge. 

^ An analysis of current tests of reading comprehension indicatec^ that 
the text passages are likely to draw broadly from knowledge of the world. 
This feature, v/hen combined with implications from the minimal comprehension 
principle, learls to^the conclusion that some of the variability in reading 
comprehension test performance is attributable to differences in prior knowl- 
edge, ratffer than differences in reading skills, £er se . This conclusion 
leads to a farther conclusion; namely, that failure on a test of comprehension 
could be attributable to lack. of prior knowledge needed to interpret a text 
.message, or to lack of reading skills. Most tests of reading comprehension 

currently in use do riot a /low one to distinguish between these two reasons 

/ 

for failure. 

The inability of a test to distinguish between failure due to lack of 
critical prior ^knowledge, and failure due to lack of reading skills, has 
little bearing on the utility of the test as a predictive instrument. However, 
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this feature g/eatly restricts the use of the test in situations where one 
-wants to either diagnose reading difficulties or assess educational gain. 
In fact, one conclusion drawn from the implications of the minimal compre- 
hension principle is that reading Comprehension tests currently in use may 
be insensitive to gains in reading skills among disadvantaged segments of 
our population. , 

Having argued that performance on current tests of heading comprehension 
was influenced, by variability in prior knowledge and reasoning ability, we 
went on to suggest that an optimal test for the purposes of diagnostics and 
< evaluation would reduce, these sources of variability to a minimum. This could 
be done by matching prior knowledge to the topical content of test passages, 
and by using assessment techniques which are relatively insensitive to dif- 
ferences in reasoning and inferential ability. 

Many of tfie suggestions we have made in this, article should be subject 
to empirical verification. While we believe there is ample evidence support- 
ing the minimal comprehension principle, there is not evidence that extent 
of prior knowledge makes an independent contribution to performance on read- 
ing comprehension tests. Experiments should be conducted to test this pos- 
sibility. If it were determined that there v;as an effect for prior knowledge, 
further research, along the lines suggested in this paper, would be needed 
to determine if those effects could be reduced. 

And finally, research is also'needed on the issue of whether reasoning 
ability can be reduced as a source of variability in reading comprehension 
test performance. We have suggested several techniques for assessing 
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comprehension which should be less respo'nsi)/e to differences in reasoning 

/ 

} I 
ability, but research is needed to establj^sh' if this is so; 
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Figure Caption 

Figure 1. A framework for viewing the comprehension process. 
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